Skip to content

feat: expose score_breakdown in StepEfficiencyMetric#2705

Open
nuthalapativarun wants to merge 1 commit into
confident-ai:mainfrom
nuthalapativarun:feat/2313-step-efficiency-score-breakdown
Open

feat: expose score_breakdown in StepEfficiencyMetric#2705
nuthalapativarun wants to merge 1 commit into
confident-ai:mainfrom
nuthalapativarun:feat/2313-step-efficiency-score-breakdown

Conversation

@nuthalapativarun

Copy link
Copy Markdown
Contributor

Summary

Populates metric.score_breakdown in StepEfficiencyMetric after each evaluation. The breakdown maps each agent step to its necessity verdict and reason, giving users per-step visibility into which actions were efficient and which were redundant.

Changes

  • deepeval/metrics/step_efficiency/schema.py — added StepAnalysis model and an optional steps: List[StepAnalysis] field to EfficiencyVerdict
  • deepeval/metrics/step_efficiency/template.py — extended the prompt to request per-step analysis alongside the overall score
  • deepeval/metrics/step_efficiency/step_efficiency.py — populate self.score_breakdown from efficiency_verdict.steps in both sync and async paths
  • tests/test_metrics/test_step_efficiency_metric.py — added TestStepEfficiencyScoreBreakdown with two no-API-key unit tests verifying correct score_breakdown population

Test plan

  • python -m pytest tests/test_metrics/test_step_efficiency_metric.py::TestStepEfficiencyScoreBreakdown -v — passes (no API key required)
  • python -m black --check deepeval/metrics/step_efficiency/ — passes
  • python -m ruff check deepeval/metrics/step_efficiency/ — passes

Example

metric = StepEfficiencyMetric()
metric.measure(test_case)

print(metric.score)          # e.g. 0.5
print(metric.score_breakdown)
# {
#   "fetch_data":     {"necessary": True,  "reason": "Required to retrieve input."},
#   "extra_llm_call": {"necessary": False, "reason": "Duplicate; first response was sufficient."},
# }

Fixes / Closes

Closes #2313

@vercel

vercel Bot commented May 29, 2026

Copy link
Copy Markdown

@nuthalapativarun is attempting to deploy a commit to the Confident AI Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose metric results of Step Efficiency

1 participant